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Abstract 

Stochastic control problems related to optimal advertising under uncertainty are 
considered. In particular, we determine the optimal strategies for the problem of 
maximizing the utility of goodwill at launch time and minimizing the disutility of 
a stream of advertising costs that extends until the launch time for some classes of 
stochastic perturbations of the classical Nerlove- Arrow dynamics. We also consider 
some generalizations such as problems with constrained budget and with discre- 
tionary launching. 

Key Words: optimal advertising under uncertainty. Bellman equation, control with 
discretionary stopping. 

1 Introduction 

We consider the optimization problem faced by a firm that, while advertising a product 
prior to its introduction to the market, wants to determine the optimal advertising policy 
for the maximization of the product image (also called goodwill), and the minimization of 
the total discounted cost. We shall also consider the problem of optimizing the launching 
time, thus allowing the firm to decide at its discretion to stop the advertising campaign 
and start selling the product. 

This type of problems can be traced back at least to Nerlove and Arrow 22j, who 
proposed to model the stock of advertising goodwill x{t) at time t > as 

xt = ut- pxt, Xo = X >0, (1) 

where ut is the rate of advertising expenditure, /) > is a factor of deterioration of 
product image in absence of advertisement. The optimization problem for a firm that 
seeks to maximize awareness of its product at a given launch time T > and to minimize 
its advertising effort until T could be formulated as a multi-objective program of the 
type 

T 



supE[e-'=^99(xT)], inf / e-^*h{ut)dt 



subject to the dynamics where c > is a discount factor, 99 : M ^ M is a re- 
ward function, /i : M — > M is a loss function, and U is the set of measurable functions 
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u : [0, T] — > U, with U a closed subset of M-(_. Following a standard procedure in 
multi-objective optimization (see e.g. Zeleny |2Hj), one takes a weighted average of the 
objectives in (j^J and obtains the deterministic optimal control problem 



where 70 is a positive constant. Sufficient conditions for the problem to be well posed are, 
e.g., that (/? is concave and continuous, h is convex and continuous, and U is compact. 
The conditions are also meaningful from an economic point of view, as it is customary to 
use concave (increasing) utility functions as measures of reward, and convex (increasing) 
loss functions. In the simplest case, one could take h(u) = u, so that the second term 
in ^ coincides with the total discounted advertising expenditure. This deterministic 
optimal control formulation has been extended by many authors to account for delay 
effects, non-linearity in the response to advertisement, and many other factors. For a 
recent work on the subject, which also contains a list of related references, we refer to 
Buratto and Viscolani '7]. 

On the other hand, less work has been devoted to the case of stochastic evolution of 
goodwill level: for a few examples of works in this direction, we refer to the survey by 
Feichtinger, Hartl and Sethi and references therein, and to the more recent papers of 
Grosset and Viscolani jl3j and Buratto and Grosset 0. The emergence of randomness 
in the dynamics of goodwill is quite natural for several reasons: one may think, for 
example, that random fluctuations in the goodwill level are the effect of external factors 
beyond the control of the firm, or that noise enters through the control, since the effect 
of advertising may be partly uncertain (see section |21 for a detailed discussion) . 

In this work we introduce some stochastic extensions of the classical model of Nerlove 
and Arrow and study related optimization problems. We do not aim at maximum 
generality, instead we focus on models whose special structure allows us to obtain explicit 
solutions. In particular, in section ^ we propose a stochastic extension of the Nerlove- 
Arrow dynamics and motivate it by marketing assumptions. We formulate a rather 
general problem of optimizing an objective function that weighs (a function of) product 
image at a fixed time and the cumulative cost of advertising effort, and we construct a 
nearly optimal advertising strategy. The special case of linear reward of goodwill and 
linear cost of advertising effort is considered in section 3: the special structure of the 
problem allows one to obtain the value function and the optimal policy in closed form, 
and to consider more general problems of advertising with a limited budget. In section 4 
we study another case where explicit solutions can be obtained, i.e. the case of quadratic 
reward of final goodwill and quadratic cost of advertising. Under these assumptions we 
also explicitly solve in section 5 a problem of optimal advertising with discretionary 
stopping to reach a target level of product awareness. We conclude suggesting some 
problems not addressed in this paper. 

2 Stochastic models for goodwill dynamics and related op- 
timization problems 

Let xt be the level of product image at time t, < t < T, where T > is the end of 
the planning horizon (the time at which the product will be launched). We postulate 




(3) 
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a dynamics for xt given by the following stochastic differential equation, which is a 
stochastic perturbation of the Nerlove- Arrow dynamics (|T]): 

dxt = {-pxt + ut)dt + a{xt,ut)dwt, xq = x > 0, (4) 

where /O is a positive constant, (T : ^ M is Lipschitz continuous, and wt is a standard 
real valued Brownian motion on a filtered probability space {Q, J- ,¥,¥), ¥ = {J^t)te[o,T]- 
The control process ut models the rate of advertising effort by the firm, and is assumed 
to be measurable, adapted, and taking values in a closed convex subset U of [0, +oo). We 
will denote by il the set of controls satisfying these properties. We use gross rating points 
(GRPs) to measure advertising effort, instead of the rate of advertising expenditure, 
following a recent trend in the marketing literature - see e.g. Dube and Manchanda [H], 
Vilcassim, Kadiyali and Chintagunta P7j . 

As briefly mentioned in the introduction, there are several reasons to study stochastic 
extensions of the Nerlove-Arrow model. Early papers on the subject such as Rao 
and Raman [2^ advocated the use of stochastic models with the observation that such 
effects as copy and competitive changes would render uncertain the effect of advertising 
on goodwill. In both papers stochastic perturbations of the Nerlove-Arrow dynamics of 
the type Q were proposed, with a not depending on xt nor ut. 

While a model with only additive noise can be useful as a first approximation, it 
is reasonable to consider more general stochastic disturbances, depending also on the 
goodwill level xt and on the intensity of advertising effort ut- In particular, one could dis- 
tinguish, potentially among others, three sources of uncertainty: a "background noise", 
of additive type, due to the kind of effects indicated by Raman [2^ and Rao [25], which 
are not directly influenced neither by the popularity of the product nor by the intensity 
of advertising. A second contribution to the intensity of noise can be attributed to the 
uncertainty in the opinion of the (potential) customers that are aware of the product. 
Finally, one should take into account the uncertainty in the effect of advertising. In 
order to model explicitly such disturbances, it may be reasonable to assume that the 
intensities of the second and third type of noise just mentioned are proportional respec- 
tively to the goodwill level xt (a proxy for the number of customers that are aware of 
the advertised product) and to the level of advertising effort Ut- In particular, the noise 
component proportional to the goodwill level could also be interpreted as the effect 
of "internal influence" (also called word-of-mouth communication), due to the random 
outcome on the goodwill level of the interaction between customers who know the prod- 
uct. Similar ideas (and terminology) are extensively employed in the literature on new 
product diffusion (see e.g. Bass [21). The third source of uncertainty described above 
could be suggestively justified by attaching to each unit of advertising effort a random 
effect, so that, heuristically speaking, "noise enters the system through the control" via 

We would also like to mention that models for the evolution of goodwill expressed 
as stochastic perturbations of the Nerlove-Arrow dynamics of the type @, where the 
diffusion coefficient a depends both on xt and on ut, appeared also as diffusion approx- 
imations of models based on discrete-time Markov chains (see e.g. Tapiero [21] and 
references therein), and in the above mentioned papers |13j . [2j. 

Due to the lack of empirical studies and of theoretical papers in the marketing 
literature on the determinants of uncertainty in the dynamics of goodwill, we are led to 
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consider, in the same spirit of the cited works of Raman and Rao, a hnear specification 
of the diffusion coefficient a: 



a{x, u) = ao + cri\x\ + a2U, 

where cjo, cj2 are fixed non-negative constants. In practice, the values of these coef- 
ficients should be determined by ad hoc empirical studies and/or by specific managerial 
and marketing insights. 

A natural analog in the stochastic setting of the general optimization problem Q 
can now be formulated. Let us define the performance functional relative to strategy 
It G il as 



7;"(s,x) =E^^ 
and the value function as 



e-^^(p{xT) - [ e-^*h{ut)dt 
Jo 



(5) 



v{s,x) = supv^{s,x), (6) 

u€iX 

where h : U is bounded and < K{1 + for some positive constants 

K, m. By E^^ we mean, as usual, expectation with respect to the law of the controlled 
diffusion 

Xt = X + / {—pXr+Ur)dr+ / {ao + ai\Xr\ + <J2Ur) dwr , s < r < t < T. 
J s J s 

Our objective is to characterize the value function and to find (or approximate) strategies 
realizing the supremum in ©. Although the problem is well posed under the given 
assumptions, particularly meaningful choices from the economic point of view are (p 
concave increasing (we have in mind the utility function of a risk-averse agent) and h 
convex increasing (typical choice of cost function). 

We shall study the problem through the dynamic programming approach, i.e. through 
the study of the associated Bellman equation, which can be written as 



sup 



+L'^^-cilj-h{u) =0, ^{T,x)=ip{x), (7) 



dt 

where is the differential operator defined by 



L" = ^a(x, u)d1 + b{x, u)dx, 

with a := aa* = and b{x, u) = —px + u. 

Equation Q is (under the assumption (Tq 7^ 0) a fully nonlinear uniformly nonde- 
generate parabolic PDE, for which general results about existence of smooth solutions 
are available under additional assumptions of smoothness and boundedness of the coef- 
ficients and of the reward and loss functions. However, we can prove existence of nearly 
optimal control in a general setting that covers also the case co = 0, for which the 
Bellman equation ^ becomes degenerate. In particular we have the following result. 

Theorem 1 For any e > 0, s G [0, T], z S M, there exists a Markov strategy G il 
such that v{s,x) < v^^ {s,x) + |e. 
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Proof. We divide the proof in three steps. In the first step we introduce a sequence of 

approximating problems, in the second step we construct an optimal feedback control 

for each approximating problem. In the last step we prove that optimal controls for the 

approximating problems are nearly optimal for the original problem. 

Step 1: Let Cn £ C'o°(I^) with Jj^ (^„(x) = 1, n = 0, 1, 2 . . ., be a sequence of mollifiers. 

Define 

{pn + u, X < —n 
—px + u, —n < X <n 
—pn + u, X > n 



and bn{x,u) = bn{x,u) * Cn{x) (convolution with respect to x). Similarly, define 
anix,u) = 



ao + ail/n + a2U, \x\<l/n 
(To + cri\x\ + (T2U, 1/n < |x| < n 
(To + (Jin + (J2n, |x| > n. 



{(/?(— n), X < —n 
(p{x), —n < X < n 
(p{n), X > n, 

and(T„(x,u) = anix,u)*Cnix), fn{x) = (pnix)*Cnix). Similarly, let = h{x)*Cn{x). 

Given a sequence of functions fnix, u), we shall say that fnix, u) converges to f{x, u) in 
^ if for each i? > one has 

lim sup sup \fn{x,u) - f{x,u) \ = 0. 

'^^°°n6(7|x|<R 

By well known properties of convolution with smooth kernels, it is easy to prove that 
(T„, 6„, hn, y^n belong to and that they converge to a, b, h, ip, respectively, in J!f. 
Let us denote by x^'^'^'{n) a solution of the equation 

Xt = X + / bn{Ur , Xr) dr + / a^iUr , Xr) dWr , 
J s J s 

where itSil, 0<s<t<T,xe M+. Moreover, let us define 

rT 



''hn{ur)dr + e "'^(pnixxin)) 



(8) 



and Vn{s,x) = sup„gii<(s, x). 

Step 2: Since an{x,u) is continuous and a'^{x,u) > for all x G M and u £ U, hy 
properties of convolutions with smooth kernels, one also has a'^(x,u) > uniformly 
over X, u for n large enough. Therefore (for a fixed large n) the value function Vn is a 
C^'^([0, r] X M) solution of the Bellman equation 



sup 



vt{t^x) + -al{x,u)vxx{t,x) + bn{x,u)vx{t,x) - cv{t,x) - hn{u) 



(9) 



v{T,x) = (fnix), 



as it follows from a result of Krylov (see ^7j, p. 301 and also TT, p. 168). As it is well 
known, if the supremum in Q is attained for each (x, t) by u"(x, t), then u" := u^{xt,t) 
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is an optimal Markov strategy for the approximating problem of maximizing ©. See 
also [12], p. 169. 

Step 3: As it follows by theorem 3.1.12 in JH]) we have that x) — > v^{t,x) uni- 
formly with respect to u € il, s G [0, T], \x\ < R for each R > 0. Therefore, given e > 0, 
there exists > such that for all n > A one has — f"| < e/3 for all u G it. Let us 
first prove that \vn — v\ < e. Assume, by contradiction, that Vn < v — e (without loss 
of generality, as the case f < f„ — e is completely analogous). By definition of v, there 
exists such that \v^ — v\ < e/3. From — f " | < e/3 we also have 

\vt -v\ < \v^' -v'''\ + \v''' -v\ < ^e, 

hence v'^^ > Vn, which is absurd. Then we have proved that \vn — v\ < e. Let us now 
take n > N large enough, so that the conditions of step 2 are satisfied, and let be a 
Markov strategy such that Vn = v^f . Then we have, by the triangular inequality, 

\v'^^-v\ < Iv"^^ - v^^ \ + \v'l!^^ - v\ 
e 4 
< 3+^=3^' 

which proves the claim. The proof of the theorem is thus finished. □ 



Under more specific structural assumptions on the objective function to optimize, it 
is natural to expect sharper characterizations of the value function and of the optimal 
advertising strategy. This is the topic of the following sections. 

3 Linear reward and loss functions 

In this section we study the simplest case, with linear reward for the level of goodwill at 
time T and linear loss function of advertising effort. As discussed before, if we identify 
ut with the rate of advertising spending, then the second term in the objective function 
© can be identified with discounted cumulative advertising costs. Let us specify the 
problem in detail. Our aim is to maximize over il the functional 

v^{s,x)=¥.l. 

where ^ 

Xt = X + / {—pXr + Ur)dr+ I a{Xr,Ur) dwr, s<r<t<T. (10) 
J s J s 

In the sequel we shall set, for simplicity, 7 = 706"^^"^. Note that, due to the linearity 
on Ut of the performance functional, we can explicitly solve the optimization problem 
even without specifying the functional form of the diffusion coefficient a, as long as 
(|l()j) admits a solution. In particular, a could be identically zero, for which we obtain 
the classical deterministic Nerlove-Arrow dynamics. The interpretation of this fact is 
simply that optimizing a linear objective function as f " simply coincides with controlling 



— cT 

70 e XT 



Ut dt, 
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the "mean evolution" of our stochastic dynamics, for which we can obtain an exphcit 
expression as follows: first write 

l-T r-T 

Jo Jo 
then take expectations on both sides to get 

rT 



E[xt] = e~P^x + [ e"^(^~*)E[nt] dt, 
Jo 



where the interchange of the order of integration follows by Fubini's theorem using the 
assumption u >0. It is now easy to guess that the functional form of a will not influence 
the optimal advertising strategy, which is expected to be of the bang-bang type. This 
is made precise in what follows. From the managerial point of view, this means that 
the firm, independently of the intensity of the noise and of its dependence on the level 
of goodwill and rate of advertising spending, will do its best in terms of maximizing 
expected goodwill by simply concentrating all its advertising efforts in a specific period 
of the advertising campaign. 

Assuming U = [0, m], the Bellman equation associated to the problem of maximizing 

over il is given by 

i't+ sup {L''ij-e-''^u) = 0, ij{T,x)=jx. (11) 

ue[0,m] 

Note that one has 

sup (L'^ib - e-^'u) = I "^"^"^^ + _ , "^^^ - 

Let us consider first the case ipx > e~^*. Then (|lljl can be written as 

i/jt - {px - m)'i(jx + ^a'^ipxx - me'"^ = 0, i/jiT, x) = jx. 
We guess a solution of the form ip{t, x) = 'y{t)x + obtaining 
x-f'{t) + b[{t) - {px - m)-f{t) - me'"^ = 0, 
with terminal conditions j{T) = 7, 61 (T) = 0. Then this equation splits into 

7' (t) - p7(t) = 0, 7(T) = 7, 
with solution 7(t) = 'ye~P^'^~^\ and 

b[{t) = -mj{t) + me-"*, bi{T) = 0, 

with solution 

6i(t) = -^(l-e-^(^-*)) + ^(e-^-e-*). 
P c 

The case ^px < e"'^* is completely similar: let be the solution of the equation j{t) = 
e""^*, i.e. t=K = . Let us now solve the equation 

- pxi/jx + ^cr'^if^xx = 0, = 7(t*)x + bi{t^), 
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where the terminal condition is such that a global solution of equation is at least 
continuous. It is immediate that the solution of this equation is ipi^jx) = 7(t)x + 6i(t*), 
so that the global solution of (fTT|) is '4){t,x) = 7(t)x + where h{t) = for 
7(t) < e"*^*, and h{t) = hi{t) for ^{t) > e^^^ . It is also easy to see that h is continuously 
differentiable on (0,T). In fact, one only needs to check whether there is smooth fit 
at t*. But since h'^it) = —m'y{t) + me~^*, by definition of t=K it immediately follows 
~ 0- This also proves that ijj G C"^'^([0, T], M), hence the solution of the Bellman 
equation is the value function of the corresponding control problem, and we can 
conclude that the optimal control is given by the following bang-bang policy: 

n,(t) = |° (12) 

That is, it is optimal not to advertise until a certain point in time t*, after which it 
becomes optimal to advertise at the maximum rate. Note that, depending on 7, it could 
well be that t^, > T, i.e. it would never be optimal to advertise. This situation arises 
if the reward for improving the image of a product is small compared to the value of 
resources spent on advertisement. 

We collect the findings of this section in the following proposition. 



Proposition 2 The optimal control problem of maximizing v^{0,x) is solved by a con 

p+c 



trol of the type il^) . with = ^'^ ^ , and the corresponding value function is given 



by v{t, x) = j{t)x + b{t), where 

m 



bi{t*) t <t^ 
bi{t) t>t^ 



Using a Lagrange multiplier method, we can treat the related problem of maximizing 
the level of goodwill at time T with a certain available budget for advertising. More 
precisely, let us consider the constrained stochastic control problem 

sup E[xt], (13) 
ueM 

where 7W C il is the set of admissible controls n(-) G [0,m] satisfying the integral 
constraint 



E 



T 

ct 



e utdt 







where M is a fixed positive constant. In order for the constrained problem to be non- 
trivial, it is also necessary to assume that M < m e~'^* dt. We actually only need 
to consider controls u for which the constraint is binding, i.e. advertising policies that 
use the whole budget M. In fact, denoting by the controlled goodwill at time T, it 
is clear that ui > U2 implies E[x^^] > E[x^^], so it is never optimal to leave resources 
unused. 

Let us introduce a Lagrange multiplier A > 0, and consider the (unconstrained) 
problem 

sup E XT-x( e-'^^utdt-M] . (14) 
ue[0,m] L V-'o /J 
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Then one has 



sup E[xt] 



= sup E 
< supE 



XT — A 
XT — A 



e^'^'^ut dt - M 



e'^^ut dt - M 



where the first equality comes from the above observation that we only need consider 
controls that use the whole budget M, and the second inequality is justified by C it. 
If the unconstrained problem ()14() admits a solution ux for all A > 0, and a A^= exists such 
that E /q e-=*«A.(t) dt-M = 0, then u^, := is an optimal control for the constrained 
problem. So we proceed to solve 



supE 
Men 



-XT 



-ct 



whose solution is 



7(t) < e-=* 
lit) > e-'^ 



Ut dt 



n*(t) = 0, 



-p(T-t) 



with 7(t) = — e" 
A 

The starting point for advertisement t* is given by the solution of the equation 

7(t) = e~^, so that 

= (15) 

P + C 

We now need to show that A > exists such that 



me 



-ct 



dt = M. 



(16) 



The solution of such an equation is given by 



A* 



M 

c h e 

m 



■cT 



which is clearly positive. It is now clear how to associate to such a A* the optimal 
solution for the constrained problem. Namely, given A* we obtain the optimal switching 
time by ((T3|) . and hence the optimal control as u^{t) = mx{t>t,}j where x is the 
indicator function. 

We have then proved the following result. 



Proposition 3 The optimal advertising policy for the constrained maximization of good- 
will HL^) is given by 

'0 t<t^, 
m t > t^, 



with 



p + c 



1 



{e'"'^ + cM/m). 
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Remark 4 It follows from H16|) that the time to start advertising is given by 



and therefore we cannot simply consider unbounded controls with cumulative discounted 
cost less or equal than M, otherwise the optimal policy would be to "do infinite adver- 
tising at time T". 



4 Quadratic reward and loss functions 



In this section we assume that both ip and h are quadratic. In particular, we assume 
'^{x) = 7ox^, with 7o > 0, and h{x) = x^. That is, we consider the problem of charac- 
terizing 



v{s,x) = supE" 



e'^'^uf dt 



or, equivalently. 



f(s,a;) = inf E"^ / e uf dt - 'jx^ 
«GU ' I Jo 



where we set 7 = e "^"^70, xt follows the controlled dynamics 



(17) 



^ + / {—pXr+Ur)dr+ / {aiXr + 0'2Ur) dw. 



Xt 



and it is the set of adapted, nonnegative, square integrable controls. A peculiar feature 
of the problem is that, while the choice of the cost function h is rather standard (see 
e.g. Muller |2j|), the reward function is convex, hence representative of a risk-seeking 
firm. Such attitude toward risk could be justified, for instance, by the attempt to profit 
from the fluctuations of the goodwill level at time T (in fact, note that, grossly speaking, 
the firm aims at maximizing both the mean and the variance of xt)- Another peculiar 
feature of (p{x) = x^ is that it equally rewards positive and negative goodwill levels at 
time T . However, we shall show that under our assumptions > almost surely, hence 
the symmetry of (/J is harmless. 

In this section we assume that the intensity of the "background noise" is negligible, so 
that we can assume fio = 0. This assumption is essential in order to obtain (meaningful) 
solutions in closed form. 

The problem at hand is a linear quadratic regulator problem with indefinite costs, 
which can be solved by the methods of Ait Rami, Moore and Zhou ^ (see also Krylov 
jlSj). In particular, the generalized Riccati equation for this problem is 



P={2p- aj)P + (1 + (71^72)^ 



P2 



-ct 



+ aiP>0 



(18) 



I P{T) 



-7- 



Recall that one says that ([T7jl is well posed at s if v{s, x) > —00. By P, well-posedeness 
of ()17() at s = is necessary for the global solvability of (jlHj) . Under the assumption 
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-cT 



(tIt > 0, one can only ensure that the problem is locally well posed, i.e. that 
there exists to < T such that the Riccati equation (|18|) admits a solution in [Iq , T] . The 
following proposition gives explicit sufficient conditions on the data of the problem such 
that (|18|1 admits a unique global solution on [0,r]. 



Proposition 5 Let us define 

ai = 
a2 = 

as = 
04 = 

c = 



-2p - 2cric7^^ - 0-2"^ < 
2p + al + 2a2^ + Aaia^-^ > 
-{ai + a^^f < 
-7ofT| + 1 > 

02 — 4aia3 = + af — 4:pal, 



where the inequality 04 > is taken as an assumption. Then the following hold: 
(i) If ( > and 02 > (2[ai[a4 — C"*^^^)^; then the problem is well posed, 
(a) If C > and 02 < (2|oi|a4 — C^^'^)'^ , then the problem is well posed if and only if 



T <C 



-1/2 



Ci log ■ 



6 log — ^ — ) , 

f 9 — 04 / 



«;/iereei,2 = (-a2±C'/')/(2ai). 
(^iiij //C = anc? 02 > 2|ai|a4, then the problem is well posed, 
(iv) //C = and 02 < 2|ai|a4, i/ien i/ie problem is well posed if and only if 



T < aVM log- 



02 



+ 



20104 



2ai04 + 02 20104 + 02^ 
('wj If ( < 0, then the problem is well posed if and only if 

r< (2oi)-i( log- 



2a2C"^/^ (atan a2C^"^/^ - atan C"^/^(2oi04 + 02)) ^ 



Proof. We shall divide the proof in three steps. In the first step we introduce an 
auxiliary LQ problem whose well-posedness is sufficient for the well-posedness of (|17() . 
In the second step we rescale the auxiliary LQ problem, and in the third and last step 
we study the global solvability of its associated Riccati equation. 
Step 1: Let us prove that if 



inf E 



uf dt — 702; 



T 



> —00, 



(19) 



then ((T7)) is well posed. In fact, suppose, by contradiction, that there exists a sequence 
u{k) G il such that 



e'^^tik) dt - jxt 



-00. 
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Then one also has 



ut{k)'^ dt — 702; 



T 



-ct 



ut{k)'^ dt — 



-00, 



which contradicts (|19|) . hence our claim is proved. 

Step 2: Let us define xt = it^^xj. Then the minimization problem in H19|) is equivalent 
to 



inf E 



2 jj. 2-2 



(20) 



subject to 



Xi = (T2 + / {-pXs + ^Us) ds+ {cTiXs + Us) dWg 

Jo Jo 



Step 3: The Riccati equation for problem (|2U() is 



P={2p-al)P + 
P{T) = -70^2 
[ P{t) + 1 > 0, 



P + 1 



which can be rewritten, after the change of variable 7r(t) = P{T — t) + 1, as 

7T = aivr + 02 + asTr""^ 
7r(0) = a4 
7r(t) > 0. 



(21) 



All assertions (i)-(v) are proved simply by determining the first time to for which 7r(to) = 
0. Since the method of proof is the same for all cases, we shall show in detail only cases 
(i)-(ii). In particular, assuming C > 0) then aivr^ + a2iT + 03 = has two roots ^1 < ^2, 
and the reduced Riccati equation ((7T|) as 

TT = aiTT~^{TT - Cl)(7r " ^2)- 

Note that oi < implies ^1.^2 = 03/fli > 0. Moreover one has ^1+^2 = cL2/\ai\, and, 
by 02 > 0, < ^1 < ^2- Therefore to < cxo if and only if 04 < ^1 = ^"^af — ' which can 
be rewritten as 02 > (2|ai|a4 — C^^"^)^ ■ In order to determine to such that '/r(to) = 0, we 
shall solve (|21j) explicitly: for this purpose, note that one has 



TT 



A 



+ 



B 



with A 



- c and B — c c 



(vr - ^i)(7r - ^2) TT-^i 7r-^2' 
- ^ , hence ()21() can also be written as 



A- 



+ B- 



vr - ^1 vr - ^2 



ai. 



Integrating one gets 
-6 



7r(s) - 6 



7^(s) 



TT S 



^2 



ai(6 - ii)t, 
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and finally 

Solving for to such that 7r(to) = 0, recalling that ai(^2 — Ci) = C^^"^} gives the required 
result. □ 

Assuming that the problem is well posed, the results in imply that the optimal 
control strategy is unique and is given by the Markov policy 

with associated value function v{s,x) = P{s)x^. The optimal trajectory is then given 
by the closed-loop equation 

dxt = a{t)xt dt + c{t)xt dwt, 

with 

il + aia2)P{t) 
il + aia2)P{t) 

which admits the explicit solution 

xt = xexp(^J {a{s) — -c{s)'^) ds + J c{s)dws^- (23) 

In particular, if x > 0, then xt is a.s. positive for all t € [0,T]. If we can prove that 
P{t) < for all t E [0, T], then (|22|) will imply that the optimal strategy is positive 
for all t € [0, T]. The negativity of P is proved in the following lemma. 

Lemma 6 If P solves the Riccati equation U8\) on [s, T], then P{t) < for all t E [s, T] . 

Proof. Assume, by contradiction, that there exists to € [s, T[ such that P{to) = 0. Then 
P(t) = is a continuous solution of H18() for t > to. By uniqueness of the solution of (fTH|l 
(see H) it follows that this is also the only solution. But this contradicts the terminal 
condition P(T) = -7 / 0. □ 

We have thus proved all claims on the optimal state and control. 

Remarks, (i) It is worth noting that several sensitivities of the value function and 
of the expected optimal goodwill with respect to initial data or parameters could be 
computed as well in terms of the solution of the Riccati equation (|18() . 

(ii) If the intensity of the noise carried by the advertising is also negligible, i.e. 
we can assume C72 = 0, then the Riccati equation ((TH|) is explicitly solvable, and the 
corresponding calculations mentioned in the previous remark simplify even more. More 
details can be found in . 
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(iii) The linear-quadratic regulator approach is also useful if one wants to consider 
problems with partial observation, which are meaningful in our setting, as goodwill can 
hardly be measured without error. Suppose instead that we can observe a "noisy proxy" 
of goodwill zt specified by 

J dzt = hxt dt + g dw} 
I ^0 = 0, 

with h and g constants, and a Brownian motion independent of w. Then one is 
interested to solve problem H17|) , where il is now the set of nonnegative square integrable 
controls adapted to the filtration generated by zt, instead of xt- Thanks to the so called 
separation principle (see e.g. Bensoussan |^), this problem reduces to linear filtering 
and deterministic control on the filtered dynamics. More details will be given elsewhere. 



(24) 



5 Optimal advertising to meet a goal with discretionary 
stopping 

Problems of mixed optimal stopping and control have recently attracted attention in 
works of applied probability, see for instance Karatzas and Wang ^Sj for applications 
to portfolio optimization, Duckworth and Zervos [2], ^01 and Zervos [22] for problems 
of investment decisions with strategic entry and exit, and Karatzas, Ocone, Wang and 
Zervos |14) for a singular control problem with finite fuel. For the theory, see, e.g., 
Krylov [T^, Bensoussan and Lions jS], 0ksendal and Sulem and Morimoto j2Uj . 
One of the first works addressing the issue of finding explicit results was Benes jH] . 

In this section we find an explicit representation for the optimal control and the 
optimal stopping strategy for the case of minimizing an objective function that is the 
sum of the quadratic distance of the goodwill from a target at the (discretionary) launch 
time r and of the cumulative quadratic cost until r, assuming that the goodwill dynamics 
is of Ornstein-Uhlenbeck type. For simplicity we also assume c = 0, i.e. we consider 
the case without discounting. In order to discourage long waiting before launching the 
product, we also introduce an extra term in the objective function depending on the 
time of launching. 

Let ut = k — Xt he the distance of the goodwill level at time t from a desired target 
k. We shall find the solution to the problem 



inf e;;-; 



y {r)+Ji / U^{t)dt + J2T 



Jo 



--: vix), (25) 



where y is such that 

dyt = (/U - pyt + Ut) dt + dwt, 

with fi := pk and we have assumed, without loss of generality (in the setting of constant 
a), a = 1. Let F be the filtration generated by w. Then il is the space of F-adapted 
square integrable control processes, and S is the set of all F-stopping times. 

Note that the first term in (|25)) assigns equal costs to the events that the target 
is not reached (from below) and that it is exceeded. Our setting could be considered 
as a stylized model for the situation when a firm is launching a new product in a 
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predetermined quantity (the target k) and considers equally undesirable to undersell 
the product (xj < k) or to leave demand unmet (xj > k). 

The quasi- variational inequality associated to the mixed problem of optimal control 
and optimal stopping (|25|) can be written as 



min ( — v{x),mm{L'^v + 71 + 72) ) = 0, 

X \ u J 



where L" is the generator of the controlled diffusion y, i.e. L" is the differential operator 
defined by 

V^f(x) = \f(x)-,{y.-px-u)f\x). 
We guess a continuation region D of the type D = {x : x > xg}, where one must have 

min(L"z; + 71 + 72) = 0. 

We have ^ 

imuiL^v + 71 + 72) = Av - - — vl + 72, 

u 471 

where A is the generator of the uncontrolled diffusion, i.e. 

Afix) = ^f"ix) + ip-px)f'{x). 

Then we get 

1 2 

Av - - — + 72 = 0, X > xo 
471 

In order to linearize this ODE, we apply the Hopf-Cole transformation U{x) = e^^i , 
obtaining 

1 72 

7;Uxx + {p - px)Ux - —U = 0, X > Xo 

2 271 

In order to obtain solutions that are ordinary functions, we restrict ourselves to the 
special case ^ = p. However, one can solve the linear equation for U without this 
assumption, obtaining solutions that can be expressed in terms of special functions. 
Two linearly independent solutions are 

POO 

J X 

Then the general solution can be written asU = aiUi + a2U2, with ai and 02 arbitrary 
real constants. 

Guided by the observation that C/i is unbounded in the continuation region, we set 
ai = 0, and impose fit of U{x) = a2U2{x) to the Hopf-Cole transformation of x^ at 
the point xq, that is 

71 
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We solve now the following system of equations for the unknowns 02 and xq: 



Oi2 



026 

fOO 



00 



p{xo-fj,/pf I g-p(s-M/p)^ — g~2^^0 



P 



71 



Therefore xq is given by the solution of the following equation 

1 /"OO 

((2p + — )a; - 2^)e''(^-^/'')' / e~P^'-P/P^^ ds = 1. 
71 Jx 

In fact the solution, if it exists, is unique, because the left hand side, as a function of 
X, is increasing. Now one can also find 02 in terms of xq, so that we have a candidate 
continuation region (or equivalently an optimal stopping region), and a candidate value 
function. 

We need to show that Av{x) — {v' (x))"^ +J2 > in the region x < xq, for v{x) = x^. 
That is, we want to show that 

(2p + — )x2-2/ix- (1+72) < 0. (26) 
71 

The expression on the left hand side takes its maximum either at x = or at x = xq- 
Therefore, if Xmax = 0, condition (|26|) is trivially verified. Otherwise, if Xmax = xq, we 
have ^ ^ 

— j = U2{xo) > =, 

(2p + 7^ )xo - 2fi pyo + Vp2y2 _^ 2p 

where yo ■= ^0 ~ p) ^-nd the inequality follows by standard estimates on the error 
function. Therefore we get 

(2/j + 7f i)xo - 2/x < pyo + ^p^yl + 2p. 

After some algebraic manipulations, one finds 

1,2 1 
(2/0 H )xo - 2p—xo < 2/071 = 72. 

71 71 

Therefore, is verified if, e.g., 71 > 1. We shall assume that in the following, but 
note that this condition can be weakened. 

Finally, we need to prove that v{x) > x^ in the continuation region x > xq, or 

!_2.2 

equivalently that U{x) > e for x > xq- In order to prove this, let us consider their 

1 ^2 

ratio /(x) = U{x)e^^^ and prove that it is increasing. One has 

fix) = a2e^'^' ({{2p + — )x - 2p)U2{x) - 1 
V 71 

and since we have that {{2p + ^)x — 2p)U2{x) is increasing in x and {{2p + ^)xq — 
2/x)C/2(x) = 1, it follows {{2p + ^)x - 2/i)C/2(x) > 1 for x > xq. Therefore we have 
verified all conditions for optimality, and we summarize our findings in the following 
proposition. 
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Proposition 7 The optimal control policy and optimal stopping time r* for the prob- 
lem \2,^) . with 7i > 1 and ^ = p, are given by 



u*{yt) = argmin(L"i;(2/t) + jiu + 72; - „ 

u 2^1 

and 

n = inf{t > : yt < xq}. 



2 , A ^'(yt) 



6 Further problems 

The optimal control problems studied in this paper are limited to the case of "smooth" 
disturbances, that is, the driving noise process has continuous paths. It is meaningful 
to relax this assumption and consider also jump components in the noise, to take into 
account possible shocks to the image of the advertised product, due, for instance, to bad 
news on the product itself or similar ones, or to the introduction of superior technologies. 

One could also try to study different type of controls, namely impulse controls, or 
even combinations of classical and impulse controls. This is particularly meaningful for 
our problems, since impulse controls correspond to the so-called "pulsing advertising" 
policies that have been studied in the management and marketing literature (see 
and references therein). 
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